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crew behavior contributed to the accident. A common pattern emerged: about three quarters of the errors represented 
plan-continuation errors - that is, a decision to continue with the original plan despite cues that suggested changing the 
course of action. Features in the context that might contribute to these errors were identified: (a) ambiguous dynamic 
conditions and (b) organizational and socially-induced goal conflicts. We hypothesize that "errors" are mediated by 
underestimation of risk and failure to analyze the potential consequences of continuing with the initial plan. Stressors 
may further contribute to these effects. Suggestions for improving performance in these error-inducing contexts are 
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Abstract 

Despite efforts to design systems and procedures to support ’correct’ and safe operations in 
aviation, errors in human judgment still occur and contribute to accidents. In this paper we 
examine how an NDM approach might help us to understand the role of decision processes in 
negative outcomes. Our strategy was to examine a collection of identified decision errors 
through the lens of an aviation decision process model and to search for common patterns. 
The second, and more difficult, task was to determine what might account for those patterns. 

The corpus we analyzed consisted of tactical decision errors identified by the NTSB from a 
set of accidents in which crew behavior contributed to the accident. A common pattern 
emerged: about three quarters of the errors represented plan-continuation errors - that is, a 
decision to continue with the original plan despite cues that suggested changing the course of 
action. Features in the context that might contribute to these errors were identified: (a) 
ambiguous dynamic conditions and (b) organizational and socially-induced goal conflicts. 
We hypothesize that "errors” are mediated by underestimation of risk and failure to analyze 
the potential consequences of continuing with the initial plan. Stressors may further 
contribute to these effects. Suggestions for improving performance in these error-inducing 
contexts are discussed. 


INTRODUCTION 

FACT: Sometimes people, even experts, make decisions that turn out badly. To what degree are 
these bad outcomes a function of inadequate decision making processes? The National 
Transportation Safety Board (NTSB, 1994) has identified tactical decision “errors” in 25 of 37 
aircraft accidents in which the flight crew’s behavior contributed to the accident. What are we to 
make of these cases? Were they all events in which the crew was “unlucky” and overtaken by 
situations beyond their control? Were the decision errors simply a function of hindsight bias, in 
which the crews’ decision processes were judged to be in error after the fact, when things turned 
out badly, despite similar decisions in similar situations that turned out OK? Were the crews 
trapped in organizational contexts that predisposed them to err? Were the tools and automation 
available to them designed in ways that invited decision error? 

In this paper we will address the meaning of the concept “decision error” within the NDM 
framework. Our effort will be guided by the view that errors, in fact, may be inevitable an 
consequence of experts behaving like experts, applying their knowledge while accomplishing 
tasks, frequently following the principle of cognitive economy. We support the position of Reason 
(1997) and Woods, Johannesen, Cook, and Sarter (1994) that we must move beyond trying to pin 
the blame for terrible accidents on a culprit, seeking instead to understand the systemic causes 
underlying the outcomes. 

The goal of our effort is to reduce the frequency of negative outcomes, specifically aviation 
accidents, by a more complete understanding of factors that lead to these outcomes, including the 
possible role of decision “errors.” Our motivation is not to label or to blame, but to identify and 
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understand the dynamics involved in so-called decision errors, and to use this understanding as a 
basis tor recommending training, procedures, or systems to improve performance. 

Three specific issues will be addressed in this paper: 

1 . What is the nature of decision errors in aviation? 

2. What factors contribute to decision errors in aviation? 

3. What can be done to reduce negative outcomes in the future? 

Before addressing these issues, however, we will examine the broader issue of how to define 
decision error in naturalistic environments. 

DEFINING DECISION ERRORS: OUTCOME OR PROCESS? 

There are two major problems in identifying decision errors in naturalistic contexts. First, errors 
typically are defined as deviations from a criterion of accuracy. However, no clear standard of 
"correctness” may exist in naturalistic contexts. The "best" decision may not be well defined, as it 
often is in a highly structured laboratory task. Second, there is a loose coupling of event outcome 
and decision process, so that outcomes cannot be used as reliable indicators of the quality of the 
decision. Redundancies in the system can "save” a poor decision or error. Conversely, even the 
best decision may be overwhelmed by events over which the decision maker has no control, 
resulting in an undesirable outcome, e.g. in windshear conditions where one aircraft may land 
safely but the next be affected by windshear. These occasions may be labeled as decision error, 
but as Fischhoff (1975). Hawkins and Hastie (1990), Woods, Johannesen, Cook, and Sarter 
(1994), and others point out. there is always the danger of hindsight bias - the tendency to define 
errors by their consequences. The problem is that the observer does not know how often exactly 
the same decision process was used or the same decision was made with no negative 
consequences. Were those prior decisions also “errors”? 

Doherty (in Lipshitz. 1997) has called for a normative process model for evaluating the quality of 
decision making on-line, independent of outcomes, both to prevent negative outcomes and to avoid 
hindsight bias. However, as Lipshitz (1997) notes, there is no normative NDM process model that 
allows definitive identification of errors independent of outcomes, as in behavioral decision theory. 
Because naturalistic decisions are embedded in task performance, it may be impossible to identify 
decision errors independent of outcomes. As Zapf et al. (1994) point out, “If we want to know 
whether or not an error has occurred we have to know the user’s goal. . ..As action is feedback 
driven. . .error detection depends on knowledge about the action outcome.” 

Woods, et al. (1994) suggest that errors be reframed as a lack of system/person fit, in which 
decision errors are the inevitable result of human activity and are potentially adaptive. That is, 
successful adaptation involves expending less effort to produce more and better output. However, 
sometimes it results in overextension of usually efficient skills in new contexts. “In unstable 
conditions, ‘correct’ models cannot be specified a priori.... The only way to test one’s decisions is 
to judge the response from the environment.” Sometimes one will be correct; sometimes one will 
be wrong. Thus, the failure is in the system, not in the individual. Errors are always contingent 
on the context and method of analysis. Moreover, to the extent that naturalistic decision making is 
“hedge clipping” rather than “tree felling” (Connolly, 1988), the decision maker him- or herself 
uses outcomes as a basis for gathering information about the situation and for evaluating decisions 
that have been made. This perspective suggests that a more productive approach to identifying 
decision errors is to examine error-inducing contexts. 

ANALYSIS OF DECISION “ERRORS” IN AVIATION 

Lipshitz (1997) points out that one contribution of the NDM approach has been to develop new 
approaches to analyzing decision processes in complex dynamic contexts, namely cognitive task 
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analysis or expert models. While not normative, these models can serve as a basis for identifying 
and characterizing error: “[NDM] standards for identifying errors pertain to situation assessment, 
mental models, and sequential option generation/evaluation rather than concurrent choice.” 
(Lipshitz, 1997, p. 158). We have used an expert model approach as a basis for analyzing 
decision making in aviation. 

Decisions in aviation typically are prompted by cues that signal an off-nominal condition that may 
require an adjustment of the planned course of action. Orasanu and Fischer (1997) described a 
decision process model for aviation that involves two components: situation assessment (SA) and 
choosing a course of action (CoA). In aviation, situation assessment involves defining the 
problem, as well as assessing the levels of risk associated with it and the amount of time available 
for solving it. Once the problem is defined, a course of action is chosen based on options available 
in the situation. Building on Rasmussen (1985), Orasanu and Fischer specified three types of 
option structures: rule-based, choice, and creative. All involve application of knowledge, but vary 
in the degree to which the response is determined. 

Thus, there are two major ways in which error may arise. Pilots may (a) develop an incorrect 
interpretation of the situation, which leads to a poor decision — an SA error, or (b) establish an 
accurate picture of the situation, but choose an inappropriate course of action -- a CoA error. 
Situation assessment errors can be of several types: cues may be misinterpreted, misdiagnosed, or 
ignored, resulting in a wrong picture of the problem; risk levels may be misassessed (Orasanu, 
Dismukes & Fischer, 1993); or the amount of available time may be misjudged (Orasanu & 
Strauch, 1994). One problem is that when conditions change gradually, pilots may not update 
their situation models. 

For example, one accident that can be traced to an incorrect assessment of the situation was the 
decision by the crew of a B-737 to shut down the wrong engine. 

The crew sensed a strong vibration while in cruise flight at 28,000 ft. A burning smell and 
fumes were present in the passenger cabin, which led the crew to think there was a problem 
in the right engine ( because of the connection between the cabin air conditioning and the 
right engine ). The captain asked which engine had the problem and the first officer replied, 
"The right one. ” The captain throttled back the right engine and the vibration stopped. 
However, this was coincidental. In fact, the left engine had thrown a turbine blade and 
gone into a compressor stall. The captain ordered the right engine shut down and began to 
return to the airport. Only then did he question which engine had the problem, but 
communication with air traffic control and the need to reprogram the flight management 
computer took precedence, and they never did verify the location of the problem. The 
faulty engine failed completely as they neared the airport, and they crashed with neither 
engine running. (AAIB, 1990). 

Errors in choosing a course of action may also be of several types. In rule-based decisions 
(Rasmussen, 1986), the appropriate response may not be retrieved from memory and applied, 
either because it was not known or because some contextual factor mitigated against it. 

Conversely, an inappropriate rule may be applied, perhaps a frequently used one (the “wrong but 
strong” schema-driven approach, Reason, 1990). In choice decisions, options may not be 
considered, or constraints that determine the adequacy of various options may not be used in 
evaluating them. Relative strengths of competing goals may play a major role in the choice of 
options. In both types of decisions, the consequences and uncertainties of courses of action may 
not be evaluated. Creative decisions may be the most difficult because they involve the least 
support from the environment and candidate solutions must be invented to fit the goals and existing 
conditions. Any solution that meets one’s goal may be considered a success. 

An accident in which an inappropriate course of action was chosen in the face of fairly complete 
information about the nature of a problem occurred in Pinckneyville, IL. 
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About two minutes into a night flight in instrument conditions, a Hawker-Siddley 
commuter aircraft lost its left generator. In error, the first officer isolated the right 
generator and then was unable to restart it. This meant total loss of the ability to generate 
electrical power, needed for all instruments in the cockpit. Under the best of circumstances 
batteries might be expected to last for 30 minutes. The captain decided to continue to the 
destination airport 45 minutes away, rather than diverting. Continued use of non-essential 
electrical equipment shortened the battery life. A complete electrical failure and subsequent 
loss of fight instruments critical for IFR flight led the plane to crash. (NTSB, 1985). 

In its analysis of the 37 accidents in which crew behavior played a role, the NTSB classified 
performance errors into nine types, including tactical decision errors. Each accident involved an 
average of 7 errors, which were often linked in episodes, as illustrated in the examples above. 
Clearly, decision errors do not occur in isolation, but are most likely to occur in a context of crew 
performance that falls short of the ideal process in many ways. 

Common Types of Decision Errors in Aviation 

To explore the idea of context-embedded decision errors, we re-examined the 37 accidents 
previously analyzed by the NTSB (1994). Our purpose was to determine what kinds of decision 
errors were most common and whether there were any themes or patterns in the contexts within 
which they occurred. We chose to use this data set because all 37 accidents were reviewed and 
errors were classified by the same NTSB analysts, providing both consistency and independence. 
The 5 1 tactical decision errors in the report were first sorted by phase of flight, yielding the 
distribution shown in Figure 1 . 

* * * * Insert Figure 1 about here * * * * 

As shown in Figure 1, the most common decision errors involved (a) failures to perform a missed 
approach or (b) failures to go around in questionable weather or with an unstabilized approach. 
111656 two types of errors accounted for 19 of the 51 tactical decision errors identified by the 
NTSB. They illustrate a common theme: many were errors in which the crew decided to continue 
with the original plan of action in the face of cues that suggested changing the course of action. In 
the Hawker-Siddley example above, the tactical decision error was to continue with the flight 
despite the lack of generators to provide electrical power, cues that indicated a change of plan 
would be most appropriate. In fact, when we sorted the decision errors into those that supported 
plan-continuation from all others, we found that 38 (or almost 75%) of the errors fell into this 
category. Besides failing to go around or to do missed approaches, they included other decisions 
involving plan-continuation, such as accepting a dispatch release for an airport with water 
contaminating the destination runway or failing to de-ice again after the time limit had expired. 
These reflect the aviation malady called “get-there-itis,” a mind set to go or to continue the flight. 

In the remaining 13 errors, crews generally performed out of the ordinary actions which they did 
not have to do. For example, in the Hawker-Siddley crash, the crew reconnected the generator and 
electrical equipment, despite having only battery power. Other examples include using reverse 
thrust on pushback and rejecting a take-off above VI (take-off speed limit). 

WHY DO DECISION ERRORS OCCUR? 

Recognizing that these errors occur is only the first step. Determining why they occur is the more 
difficult task. Two thirds of the plan-continuation errors in the NTSB data base involved 
omissions, that is, failures to do something that should have been done or to take some required 
action. It might appear that crews are captured by a familiar schema in ambiguous conditions, 
which leads them to do what they normally do, that is, to carry on with the usual plan. However, 
this explanation does not account for errors of commission that also contribute to plan- 
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continuation. These are cases in which crews take actions that are out of the ordinary, such as 
attempting to have snow blown off their aircraft by using engine exhaust from the aircraft ahead of 
them. 

Bounded Rationality 

Prior efforts to identify causes of decision error have focused primarily on the cognitive aspects: 
‘buggy’ mental models or misapplication of knowledge due to limitations in human cognitive 
processing (Reason, 1990, Rasmussen, 1985). In his analysis of performance in several 
naturalistic domains, Klein (1993) identified three causes of decision error that are consistent with 
a “bounded rationality” explanation (Simon, 1957): lack of information, lack of knowledge, and 
failure to simulate outcomes 

• Lack of Information may stem from poor system design, such as when automation does not 
provide adequate diagnostic information or feedback. 11115 is not really a cognitive problem, but 
expertise may lead a decision maker to question the adequacy and accuracy of available 
information. For example, the crash in which the flight crew shut down the wrong engine resulted 
in part because the information about which engine had the problem was poorly displayed. 

• Lack of Knowledge: Knowledge and expertise are specific to situations. Lack of knowledge 
can lead to misdiagnosis of a problem or to choice of a poor option. For example, one may 
suspect lack of knowledge was the basis for a crew deciding to use engine exhaust from the aircraft 
ahead of them to blow snow off their wings. Or misassessment of risk could result from lack of 
familiarity. In several aviation accidents, lack of current knowledge seemed to play a role: While 
pilots were experienced in general, they were out of practice. Pilots had been off duty for medical 
reasons or temporary military duty (e.g., National Guard), were new to the aircraft (recently 
transitioned), or were unfamiliar with the airport or the particular approach or runway. 

• Failure to Simulate usually leads to errors in choosing a course of action. Pilots do not 
anticipate the consequences of an action (or lack thereof), especially when the situation is evolving 
dynamically. They may fail to project potential risk into the future. Failure to fully evaluate the 
consequence of an option may result from excess workload or time pressure. 

Error-Inducing Contexts 

In addition to bounded rationality, features of the situation may interact with cognitive limitations to 
induce errors in the decision making process. We suggest that the following factors may 
contribute in particular to the high incidence of plan-continuation errors. Organizational and 
social pressures create goal conflicts, which are perhaps most likely to result in decision errors 
in the face of ambiguous cues and high-risk situations. Decision errors may be mediated by 
underestimation of risk, overestimation of one’s competence to handle the situation, or failure to 
evaluate the consequences of planned actions. Factors may combine to yield highly stressful 
conditions, which further impact the pilot’s decision making ability. How they might operate is 
described below. 

• Ambiguity. Cues that signal a problem are not always clear-cut. Conditions can deteriorate 
gradually, and the decision maker’s situation assessment may not keep pace. If events occur 
infrequently, the decision maker may not have amassed the experience to recognize the signals 
associated with a change in course of action. A recurring problem is that pilots are not likely to 
question their interpretation of a situation, even if it is in error. Ambiguous cues may permit 
multiple interpretations. If this ambiguity is not recognized, the crew may be confident that they 
have correctly interpreted the problem. Even if the ambiguity is recognized, a substantial weight of 
evidence may be needed to change the plan being executed. Weather and certain system 
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malfunctions can change dynamically and pose a challenge for situation assessment, including risk 
and time assessment. 

Ambiguous conditions may have three consequences that increase the chance of decision error. 
First, because of the uncertainty in the situation, pilots do not know clearly what to do. Second, 
they may underestimate the risk, particularly when the situation is deteriorating gradually. Third, 
they may find it difficult to justify a change in plan in the face of ambiguous cues. For decisions 
that have consequences, such as rejecting a takeoff or diverting, the decision maker needs to justify 
a course of action that may entail a cost. If the situation is ambiguous, the decision is harder to 
justify than if the situation is clear-cut, which may work against a decision to change the plan. 
Ambiguous cues create conditions that may increase the likelihood of decision error, either because 
of inappropriate situation assessment, risk assessment, or difficulty in justifying a change of plan. 

• Organizational and Social Factors. An organization’s emphasis on productivity may unwittingly 
set up goal conflicts with safety. For example, on-time arrival rates are broadcast to the public, 
“We’re Number One in Arrivals!” One airline went so far as to pay passengers $ 1 for each minute 
their flight was late — until a crew flew through instead of around a thunderstorm and crashed 
(Nance, 1986). Companies also emphasize fuel economy and getting passengers to their 
destinations rather than diverting. Cutting comers or taking risks may be tolerated or subtly 
encouraged. 1 

The crash of a US Air Force plane carrying Secretary of Commerce Ron Brown to a meeting in 
Dubrovnik appears to be a case in which organizational factors pressured the crew to take risks, 
perhaps against their better judgment. The aircraft they were flying was not legal for the approach 
into Dubrovnik (it needed two Automatic Direction Finder radios, but had only one); the crew was 
not adequately trained to read the charts for the instrument approach; and the Command allowed 
them to use civilian rather than military charts, which were not authorized. Pilots typically are 
rewarded for a “can do” attitude; saying “can’t do” may be difficult. Being a highly visible 
mission, expectations were strong for the crew to fly it, despite these handicaps. 

Social factors may also create goal conflicts. Implied expectations among pilots may encourage 
risky behavior or may induce one to behave as if one is an expert, even in the face of ignorance. 
This may result in unwillingness to admit that one does not know something, is unfamiliar, is 
uncertain, is lost (c.f., the wives’ lament, “Why won’t you stop to ask directions”?). For 
example, a runway collision in near zero visibility (due to fog) resulted when one aircraft stopped 
on an active runway, the crew not realizing where they were. The captain was unfamiliar with the 
airport, and was making his first unsupervised flight after a long period of inactivity. The first 
officer boasted of his knowledge of the airport but in fact gave the captain incorrect information 
about taxiways. Rather than questioning where they were, the captain went along. Meeting social 
and organizational goals often appears to outweigh safety goals, especially in ambiguous 
conditions. 

Ambiguous cues and organizational and social factors may not in themselves be sufficient to cause 
decision errors. However, when the decision maker’s cognitive limits are stressed, these factors 
may induce errors in certain contexts. Errors may be mediated by underestimation of the risk 
inherent in a situation, overconfidence in one’s ability to cope with the situation, or failure to 
evaluate the consequences of planned actions. 

• Underestimating risk. In several accidents, the crew clearly was aware of cues that should have 
signaled a change in course of action, but appeared to underestimate the level of risk associated 
with them. For example, when approaching Dallas for landing, the first officer of an L- 1011 


1 Organizational decisions about levels of training, maintenance, fuel usage, keeping schedules, etc. may set latent 
pathogens that undermine safety, in the face of vocal support for a "safety culture." (Reason, 1990). 
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commented on the lightning in the storm lying on their flight path. Yet they flew into it and 
encountered wind shear. We know that threat is important to pilots, because potential risk was the 
dominant dimension considered by captains from several airlines when making judgments about 
flight-related decision situations (Fischer & Orasanu, 1995, 1997). Why then do crews appear to 
underestimate risk 2 in potentially critical situations? 

One explanation is grounded in their experience. If similar risky situations have been encountered 
in the past and a particular course of action has succeeded, the crew will expect also to succeed this 
time with the same response, e.g. landing at airports where conditions frequently are bad, such as 
in Alaska. Given the uncertainty of outcomes, in many cases they will be correct, but not always. 
Reason (1990) calls this ' frequency gambling Contrary to allegations that decision makers fail to 
use base rates, this would appear to be an appropriate use of them. 

Support for this notion can be found in several sources. In his investigation of mid-air collision 
accidents, Schuch (1992) found that they mainly involve experienced pilots. He concluded that 
because experienced pilots have made repeated flights without an incident, they become 
desensitized and stop scanning the sky. Hollenbeck et al. (1994) found that past success 
influences risk taking behavior. Baselines become misrepresented over time as a situation becomes 
familiar and the individual becomes more experienced. Sitkin (1992) argued that if you only have 
good experiences you have no baseline by which to determine when the situation is taking a turn 
for the worse. 

A second factor that may increase apparent tolerance of risk is framing. Studies have found that 
people tend to be risk averse when outcomes are framed as possible gains, but risk seeking when 
outcomes are framed as potential losses (Kahneman & Tversky, 1984). This raises the question of 
whether deteriorating situations that imply a change of plan, for example, to divert or to go around, 
are seen as loss situations and therefore promote risk-seeking behavior. Pilots may be willing to 
take a risk with safety (a possible loss) to arrive on time (a sure benefit). 

• Consequences are not anticipated. Ambiguity and goal conflicts both imply that multiple courses 
of action should be entertained, first, because different readings of ambiguous cues might lead to 
different actions, and second, because different actions might be needed to satisfy competing 
goals. Reaching decisions under these circumstances requires projection and evaluation of the 
consequences of the various options. If pilots are under stress, they may not do the required 
evaluations. Stress limits the decision maker’s ability to project the situation into the future and 
mentally simulate the consequences of a course of action. By constraining attention and working 
memory capacity, stress can interfere with consideration of multiple hypotheses, thus limiting 
evaluation of options (Stokes et al„ 1992, Stokes & Kite, 1994). Under stress, people often 
behave as though they are under time pressure, when in fact they are not. Under stress, decision 
makers often fall back on their most familiar responses, which may not be appropriate to the 
current situation. For example, one captain retarded power on all four engines shortly after take- 
off when he sensed a vibration through the aircraft. Reducing power during this critical phase of 
flight was just the wrong thing to do so close to the ground. The action might have been 
appropriate in a different context. In fact, a report submitted to the Aviation Safety Reporting 
System (ASRS) described how a vibration was eliminated by reducing power (the vibration being 
due to a loose trim tab). However, that aircraft was in cruise and suffered no ill effects. Because 
most abnormal events tend to be quite infrequent, the correct responses may not be familiar to the 
crew. Stress may interfere with recognition of the inappropriateness of wrong responses, such as 
shutting down the wrong engine. 


: Risk refers to the likelihood of a negative outcome occurring as a result of a problem, or the likelihood of various 
outcomes associated with various actions or no action on the part of the crew. 


5/22/98 


8 



Aviation Decision Errors 


Certain phases of flight induce higher levels of stress, namely those in which time is limited, 
workload is heavy, and there is little room for recovering from an error. These tend to be take-off 
and landing situations. Mistakes during these periods may have serious consequences. However, 
dangerous events can occur in cruise as well, and some of these must be responded to quickly, 
such as traffic conflicts, loss of an engine (in a two-engine aircraft), fire, turbulence, or rapid 
decompression. Other serious conditions may need attention, but may permit more thinking time 
(e.g., fuel leaks, hydraulic, electrical or communication failures, or passenger problems). 

Stressful conditions may induce the use of decision heuristics, such as satisficing, which often 
yield satisfactory results. However, when cues are ambiguous and goal conflicts exist, those may 
be just the situations that require more thorough analysis. 

SUMMARY AND DISCUSSION 

We have identified several error-inducing contexts, features of the environment that are likely to 
induce error when combined with the cognitive limitations of the human information processing 
system. These are ambiguity and goal conflicts produced by organizational and social factors. 
Because of their inherent uncertainty, they require more thorough analysis than unambiguous 
situations in order to avoid possible error. If they occur when time is limited, workload is high, or 
other stressors are present, pilots may not have the resources to devote attention to them. One 
particular type of error frequently results: plan continuation errors, which are characterized by 
pressing on with the original plan in the face of cues that suggest a change would be warranted. 

We suggest that these errors are mediated by an underestimation of the risk associated with the 
problem and a failure to evaluate the possible consequences of continuing or adopting a different 
course of action. Clearly, more cognitive effort is needed to revise one’s understanding of a 
situation or to consider a new course of action than sticking with the original plan whose details 
have already been worked out. Given that pilots, like most people, are “cognitive misers,” they 
tend to avoid such changes. It appears that evidence must be unambiguous and of sufficient 
weight to prompt a change of plan. 

We offer the following suggestions about what might be done to prevent or to catch such errors, 
thereby preventing negative outcomes such as accidents. 

• Can situations be rendered less ambiguous? Integrated flight displays that present accurate 
system information, as well as up to date information on unpredictable, dynamic attributes of the 
situation, such as weather and traffic, may render situations less ambiguous. Trend information 
appears most critical. Information must be relevant as well as available, hence the information 
provided should be critical to pilots in their particular phase of flight. 

• Can organizational goal conflicts be reduced? To reduce the pressures on pilots to continue with 
the original plan requires their organizations to recognize the bind they put their pilots in. While it 
is not realistic to expect that companies will reduce their concern with productivity, they must be 
willing to stand behind their pilots who take a safe course of action rather than a riskier one, even if 
there is a cost associated. 

• Can pilot risk assessment be made more accurate? Addressing the mediating factors of a tactical 
decision is difficult, simply because they are context dependent, which makes predictions difficult. 
Perhaps aircraft system diagnostics could have risk estimates attached. Weather and traffic 
displays already have some risk information (color coding of weather severity; Traffic/Collision 
Avoidance System or TCAS warnings for traffic). Aids providing judgments of time available for 
assessment or action require predictive models. They could indicate how long it will take for a 
condition to degrade to a critical state, e.g. fuel consumption or reserve battery life span, how soon 
weather will improve or a storm hit, or when traffic will dissipate in target region. 
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A factor that is specific to aviation (and a few other high-risk environments) is that pilots are 
making decisions not only about a situation external to themselves, but about a system of which 
they are a part. They not only make the decision, they implement it. Thus, situation assessment 
must include judgment of a pilot’s capability and skill, stress levels, workload, and available time. 
If a pilot decides to continue with an approach in the face of less than ideal weather, it is that pilot 
(or the co-pilot) who must maneuver the aircraft. So, the judgment about whether or not to 
continue the flight is not only about the weather conditions and the aircraft capability, but also 
about the pilot’s own ability to handle the situation. In addition, the pilot bears the consequences 
of his/ her decisions. As it has been said, the pilot is the first person at the scene of the accident. 

• Can pilots be induced to evaluate the consequences of their planned actions? Another area of 
training and support lies in assisting pilots with strategies for choosing a course of action. Aiding 
may consist of prompting crews to consider constraints on options prior to acting, to consider the 
disadvantages of the selected option, and the likelihood of various outcomes. Klein (1997) 
included such recommendations in a training model for military decision making. Tools for doing 
"what if' reasoning and managing multiple hypotheses may also be helpful, encouraging forward 
thinking. Means et al. (1993) stress the importance of making decision makers aware of the worst 
case scenario and training them to manage this. 

Aviation is a domain in which errors are inevitable (Woods, et al., 1994). Therefore, the aviation 
system needs to support error catching, as well as error prevention. Just as equipment is designed 
with redundancies to prevent a total system failure, the crew should build on its redundancy to 
assist error catching. These methods include monitoring each other as well as the situation, and 
communication to support building shared problem models. The importance of error catching was 
recently illustrated by an incident in Japan. A C- 130 Air Force transport was erroneously de-iced 
with washing fluid, adding to the problem rather than removing it. Only the refusal of the ramp 
attendant to remove the chocks from under the wheels, due to the visible icicles, prevented the 
crew from attempting to take-off. 

In summary, our re -analysis of the NTSB accidents emphasized that the types of decisions and 
errors are specific to this complex domain. We must be cautious about generalizing across 
domains. At a definitional level, there are common elements: explicit or implicit situation 
assessment and option choice, usually with sequential consideration of options, in a knowledge- 
rich environment. Human capacities are limited, but are also the source of strength and flexibility 
which can be used both to prevent and to catch errors. Providing aids that support decision 
making depends on understanding the context - how the specific error-inducing features of the 
situation interact with mediating factors and how tools and strategies will interact with these 
features of the domain. 
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FIGURE 1 
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